TESTING PLAYBOOK

ABC Member Engagement Reporting Module — Testing Playbook

Master engagement point-based scoring, criteria configuration, report generation, and data-driven member outreach and recognition strategies

The Business

Member Engagement Reporting enables chapters to measure and track member company and individual participation using a point-based scoring system. Staff configure engagement criteria, generate reports, and identify disengaged members for outreach while recognizing highly engaged participants.
📊 What Does Member Engagement Reporting Do

Member engagement tracking is critical for chapters to understand which companies and individuals are actively participating and which are at risk of declining participation. Traditional approaches rely on manual tracking—spreadsheets, anecdotal observations, and institutional memory. This creates blind spots, inconsistent analysis, and missed outreach opportunities.

Member Engagement Reporting automates this. Chapter staff defines engagement criteria (event attendance, committee participation, sponsorship, donations, etc.), assigns point values, and sets minimum thresholds. The system automatically scores member participation across events, committees, and groups. Reports show company-level totals and breakdowns by individual member, with flexible filtering and Excel export for analysis.

Business value: (1) Data-driven decisions—chapters see exactly who is engaged and who needs attention, (2) Targeted outreach—staff can prioritize members below minimum engagement thresholds for personalized follow-up, (3) Recognition—highly engaged members and companies are identified for awards and acknowledgment, (4) Program effectiveness—chapters understand which initiatives drive participation, informing future programming decisions, (5) Eliminated manual tracking—no more spreadsheets or conflicting records.

Who Uses It and Why It Matters

Chapter Staff (L2/L3 users): Configure Report Settings (year, date range, filters), manage standard and custom engagement criteria, assign ME Classifications to events/committees/groups, and generate reports.

Executive/Board Members: Use reports to understand overall membership health, identify at-risk accounts for board discussion, and recognize top performers at recognition events.

Membership/Engagement Leads: Run reports filtered by company status and engagement level, then execute targeted outreach (emails, calls, personal invitations) to re-engage inactive members.

Why it matters: Without engagement visibility, chapters miss renewal opportunities (disengaged members leave), fail to recognize contributors (missed relationship-building), and lack evidence for programming ROI (difficult to justify events and initiatives).

🔗 Key Confluence Pages and Related Tickets

Primary Documentation:

Test Coverage:

  • 16 Confluence line items across 2 pages documenting all feature requirements
  • 29 Jira tickets covering configuration, reporting, and integration testing
QA Focus for The Business Section

Understand that Member Engagement Reporting is a strategic tool for chapter retention and growth. It transforms subjective impressions of member activity into objective, reportable data. QA should verify that the system is flexible enough to accommodate diverse chapter engagement models (event-focused vs. committee-focused vs. mixed) while maintaining consistent scoring and accurate data.

How It Works

A detailed walkthrough of configuration, criteria management, classification assignment, and report generation.
⚙️ Step 1: Configure Report Settings

Chapter staff navigates to Member Engagement Reporting module and begins configuration. First step: Report Settings.

Year Selection: Staff selects the data collection year (e.g., 2024). Year automatically populates date range to 1/1 through 12/31, but dates are editable (e.g., staff can run mid-year reports for Jan-Jun).

Include Annual Criteria Checkbox: Controls whether criteria marked with "Annual Designation" are included in this report run. Annual criteria can be met at any point during the year (e.g., "Attended annual conference" counts even if attended in December). Non-annual criteria require completion within the specified date range.

Company Filters: Optional filters for paid status, company size, chapter role, etc. Allows staff to run engagement reports for specific member segments.

Settings are saved for this report run only—staff must re-enter on next report generation.

✏️ Step 2: Manage Standard Criteria (17 Pre-Defined)

The system provides 17 pre-defined Standard Criteria that cannot be deleted, only activated or deactivated:

  1. Reason for Joining
  2. Leadership Committee
  3. Attend Membership Events
  4. Sponsor Event
  5. Apprenticeship/Craft Training
  6. Attend Safety Events
  7. STEP
  8. AQC
  9. Attend Mgmt Ed Events
  10. Committee Participation
  11. Recruit Members
  12. PAC Donors
  13. Voluntary Contributions
  14. Referral Request
  15. Discount Program
  16. Partner Program
  17. Advertisers

Configuration for Each Criterion: Staff sets Point Value (number of points awarded), Min # (minimum occurrences before points awarded, typically 1), Multiple Occurrence checkbox (unchecked = single award; checked = accumulate per occurrence), and Annual Designation checkbox (marks criteria valid for any time in year).

Subsystem Indicators: Read-only checkboxes showing which subsystems (Events, Committees, Groups) each standard criterion applies to. Staff cannot modify—determined at system design.

Young Professional criterion: Separate standard criterion for tracking Young Professional engagement.

🛠️ Step 3: Manage Chapter-Specific Criteria (Unlimited Custom)

Chapter staff can create unlimited custom criteria beyond the 17 standard ones. Examples: "Attended Q1 Marketing Forum" (custom event), "Served on Scholarship Committee" (custom committee), "Donated to Charity Drive" (custom activity).

Configuration: Name (serves triple purpose: column header, ME classification name, and identifier), Point Value, Min #, Multiple Occurrence checkbox, Annual Designation checkbox, and optional Event/Committee/Group checkboxes (marking which subsystems this criterion tracks).

Create/Edit/Inactive: Staff can add new criteria at any time, edit existing criteria (changes apply to future reports), and inactivate criteria (hidden by default via Hide Inactive toggle). Inactivating does NOT delete—if reactivated, historical assignments persist.

Hide Inactive Toggle: Default ON. Hides inactive chapter-specific criteria from display. Staff can toggle to ON to view full list including inactive. Standard criteria cannot be hidden.

🏷️ Step 4: Assign ME Classifications to Events, Committees, Groups

ME (Member Engagement) Classification is a SEPARATE field from standard Event/Committee/Group Classification. This is critical: ME Classification is used exclusively for engagement reporting, while standard Classification is used for other module purposes.

Assignment Process: When creating or editing an Event, Committee, or Group, staff assigns an ME Classification value. Examples: "Attend Membership Events" classification applied to member-only events, "Committee Participation" to active committees, "Sponsor Event" to events where sponsorship is tracked.

Match with Criteria: ME Classification names match Standard and Custom Criteria names. System automatically scores any individual who participates in an event/committee/group with that classification—points accumulate based on criteria settings (Multiple Occurrence, Min #, Point Value).

Independence: Standard Classification (used for event type, registration rules, etc.) is independent. An event can have Standard Classification "Annual Conference" and ME Classification "Attend Mgmt Ed Events" simultaneously.

📈 Step 5: Generate Report (Company or Individual Level)

Report Output Options: Staff selects default Company Level Output (aggregate scores by company) or Individual Level Output (individual member scores grouped by company).

Company-Level Report: Columns include Company Name, Total Points, Paid Status, and individual criteria point columns. Total Points = company-level points + SUM(all individual member points). Useful for board discussions and partnership reviews.

Individual-Level Report: Columns include Company Name, Individual Name, Total Points (individual only), and criteria breakdowns. Individuals grouped by company. Useful for targeted outreach and recognition.

Real-Time Generation: Reports are generated on-demand based on current data. If activities occur after report runs, subsequent report regeneration includes new data.

🔍 Step 6: Filter, Sort, and Export Results

Sorting: Report results are sortable by all columns. Default sort by Total Points descending (highest engagement first). Staff can re-sort by company name, individual name, specific criteria, or paid status.

Filtering: Optional filters for company status (active/inactive), paid status, point range ("companies with 0-100 points", "individuals with 50+ points"), and individual criteria ("members who attended events but no committee participation").

Export to Excel: Full report data can be exported to Excel format, preserving all columns and sorting/filtering applied. Useful for offline analysis, board presentations, and CRM import.

QA Focus for How It Works

Test the complete workflow: configure settings → manage criteria → classify events → generate reports → export. Pay special attention to (1) date range logic and annual designation behavior, (2) ME Classification independence from standard Classification, (3) point accumulation with Multiple Occurrence flag, (4) company total calculations (company points + sum of individual points), and (5) persistence of criteria and filter state across sessions.

Glossary

Key terminology for Member Engagement Reporting. Ensure consistent terminology across UI, help text, error messages, and documentation.

ME Classification

Dedicated engagement classification field separate from standard Event/Committee/Group Classification. Assigned to events, committees, and groups to indicate what engagement metric they contribute to. Examples: "Attend Membership Events", "Committee Participation", "Sponsor Event".

Standard Criteria

17 pre-defined engagement measures that cannot be deleted, only activated/deactivated. Fixed set across all chapters. Examples: Attend Membership Events, Leadership Committee, PAC Donors, Committee Participation.

Chapter-Specific Criteria

Custom engagement measures created by chapter staff. Unlimited quantity. Can be created, edited, inactivated, or reactivated. Examples: "Attended Q1 Marketing Forum", "Served on Scholarship Committee".

Multiple Occurrence

When checked, points accumulate per occurrence (e.g., attending 3 events = 3x points). When unchecked, points awarded once regardless of occurrence count (e.g., attending any event = single award).

Annual Designation

Marks criteria that can be met at any point during the data collection year. Example: "Attended annual conference" counts if attended in January or December. Controlled by "Include Annual Criteria" checkbox in Report Settings.

Min # (Minimum Instances)

Minimum occurrences required before points are awarded. Example: if Min # = 3 for "Committee Participation", member must attend 3+ committee meetings before earning points.

Company Total Points

Company-level points (if applicable) plus the SUM of all individual member points from that company. Used in company-level reports to show overall engagement by account.

Paid Status

Whether a company is current on membership/dues payments. Used for optional filtering in Report Settings and display in reports.

Configuration Persistence

Criteria settings (Standard, Chapter-Specific, ME Classifications) are saved and persist across sessions. Report filters and settings are per-run and do not persist unless explicitly saved.

Hide Inactive Toggle

Default ON. Controls visibility of inactive chapter-specific criteria in the UI. Does not affect reports—inactivated criteria still calculate if data exists. Standard criteria cannot be hidden.

Subsystem Indicators

Read-only checkboxes (Event, Committee, Group) showing which subsystems each Standard Criterion applies to. Cannot be modified by chapter staff—determined at system design.

Data Collection Year

The year for which engagement data is being collected and reported. Auto-populates date range 1/1-12/31 but dates are editable for mid-year or custom reporting periods.

QA Focus for Glossary

Ensure terminology is consistent across all UI surfaces. If staff sees "ME Classification" in one place and "Engagement Classification" in another, flag as inconsistency. Critical distinction: ME Classification vs. standard Classification must be clear in UI labels and help text.

Test Strategy

Comprehensive testing approach for Member Engagement Reporting, covering configuration, criteria management, report generation, classification assignment, and integration with events, committees, and groups.
⚙️ Configuration Testing

Report Settings: Test year selection, date range edits, company filters, and annual criteria checkbox. Verify all settings save correctly and filter report output accurately.

Standard Criteria: Verify all 17 pre-defined criteria are present and cannot be deleted. Test activation/deactivation, point value changes, Min # and Multiple Occurrence flag edits. Verify changes persist across sessions.

Chapter-Specific Criteria: Test creation of custom criteria with all optional fields. Test editing criteria names (verify ME Classification name updates). Test inactivation (not deletion). Verify Hide Inactive toggle hides/shows as expected.

Validation: Test field-level validation: numeric point values, positive Min # values, required name fields, duplicate name prevention.

📊 Report Accuracy Testing

Company-Level Points: Verify company total = company-level points + SUM(all individual member points). Test with multiple individuals under same company.

Individual-Level Points: Verify individual points calculated correctly based on ME Classifications assigned to their event/committee/group participation. Test Multiple Occurrence accumulation vs. single award behavior.

Min # Threshold: Test that points are NOT awarded if occurrence count < Min #. Test threshold boundary (e.g., Min # = 3, test with 2 occurrences, 3 occurrences, 4 occurrences).

Annual Designation: Test with annual criteria enabled vs. disabled in Report Settings. Verify annual criteria included when checkbox ON, and included regardless of date range when annual designation is set.

Date Range Logic: Test with custom date ranges (e.g., Jan-Jun). Verify non-annual criteria outside date range do not contribute points.

🏷️ Classification Separation Testing

ME Classification vs. Standard Classification: Verify that an event/committee/group can have both Standard Classification and ME Classification simultaneously without conflict. Example: event with Standard Classification "Annual Conference" and ME Classification "Attend Mgmt Ed Events" should work correctly.

ME Classification Assignment: Test assigning ME Classification to events, committees, and groups. Verify assignment persists. Test modifying ME Classification (verify updated for future reports).

No Subsystem Conflict: Verify that ME Classification on an event doesn't interfere with standard Classification-driven event type rules, registration logic, or other subsystem behavior.

📋 Scope Change Verification (10 Requirements)

Test all 10 scope change requirements from ME Reporting Scope Changes Confluence page:

  • ME Classification is separate from standard Classification
  • Standard Criteria are pre-defined and cannot be deleted
  • Chapter-Specific Criteria are user-created and can be inactivated
  • Company Total Points = company points + individual sum
  • Configuration settings persist across sessions
  • Report filters do not persist (per-run only)
  • Hide Inactive toggle controls visibility of inactive chapter-specific criteria
  • Annual Designation checkbox controls inclusion in report
  • Min # threshold enforcement
  • Multiple Occurrence accumulation behavior
🔗 Cross-Module Integration Testing

Event Integration: Test assigning ME Classifications to events. Verify event participants are scored correctly in engagement reports. Test updating event ME Classification mid-year (verify applies to future participants).

Committee Integration: Test assigning ME Classifications to committees. Verify committee members are scored. Test adding/removing committee members (verify score updates in next report run).

Group Integration: Test assigning ME Classifications to groups. Verify group members are scored correctly.

Participant Sync: Verify that participation data flows correctly from events/committees/groups to engagement scoring without data loss or duplication.

📥 Excel Export Validation

Data Integrity: Export report to Excel and verify all columns, values, and sorting match on-screen display. Spot-check company totals and individual points.

Format: Verify Excel headers, number formatting (currency for points if applicable), and readability.

Filtered Exports: Apply filters/sorting in UI, export, and verify exported data reflects filters/sorting (not full dataset).

⚠️ Edge Case Testing

No Qualifying Data: Generate report when no members meet criteria. Report should display empty or zero scores, not error.

All Criteria Inactive: Inactivate all criteria and generate report. Report should display, all scores should be zero.

Max Criteria Count: Test system behavior with very large number of chapter-specific criteria (100+). Verify performance and UI usability.

Boundary Conditions: Test Min # = 1 (single occurrence), Min # = very high value (unlikely to reach). Test Point Value = 0 (no points awarded). Test with date ranges where no participation occurred.

Membership State Changes: Create company, run engagement report, then inactivate company. Verify previous reports still show company; new reports exclude inactive company (if company filter applied).

QA Focus for Test Strategy

Member Engagement Reporting touches multiple subsystems (Events, Committees, Groups) and depends on accurate data aggregation. Test strategy should balance coverage of configuration UI with validation of point calculations and classification behavior. Pay special attention to data persistence and edge cases where criteria change mid-year.

Business Rules & Gotchas

Critical business rules and common gotchas. Expand each card to understand nuances that drive test cases and inform QA strategies.
📅 Rule 1: Year and Date Range Auto-Population

The Rule: When staff selects a data collection year, date range automatically populates to 1/1 through 12/31 of that year.

The Gotcha: Dates are editable. Staff can override to run Jan-Jun reports, Oct-Dec reports, or any custom range. System must support both.

QA Test: Set year to 2024 and verify dates auto-populate 1/1/2024-12/31/2024. Then edit start date to 4/1/2024 and verify system accepts and reports correctly for Jan-Mar exclusion.

Rule 2: Include Annual Criteria Checkbox

The Rule: When "Include Annual Criteria" is checked in Report Settings, criteria marked with Annual Designation are included in calculations. When unchecked, annual criteria are excluded.

The Gotcha: Annual designation is a separate concept from date range. An annual criterion can be met in December but still counts if checked, even if report date range is Jan-Jun.

QA Test: Create criterion "Attended Annual Conference" with Annual Designation ON. Run two reports: one with checkbox ON (criterion included), one with checkbox OFF (criterion excluded). Verify different totals.

🔒 Rule 3: Standard Criteria Cannot Be Deleted

The Rule: 17 Standard Criteria are pre-defined, fixed, and cannot be deleted by chapter staff. Only activation/deactivation is allowed.

The Gotcha: UI should provide no delete option for standard criteria. Attempting to delete (if UI glitch occurs) should fail gracefully with clear message.

QA Test: Right-click or attempt to delete a standard criterion. Verify delete option is not available. Verify UI clearly marks standard criteria as "System-Defined" or similar.

Rule 4: Chapter-Specific Criteria Are Unlimited and Inactivatable

The Rule: Chapter staff can create unlimited custom criteria. Criteria can be inactivated (hidden, not deleted). Inactivated criteria still persist and can be reactivated.

The Gotcha: "Inactivate" is not delete. Data is preserved. If staff later reactivates, historical participation data still exists. Hide Inactive toggle controls visibility, not deletion.

QA Test: Create 5 custom criteria, inactivate 2, toggle Hide Inactive ON/OFF. Verify inactive criteria disappear/reappear in list but data is preserved. Reactivate an inactivated criterion and verify it recalculates in next report.

📝 Rule 5: Chapter-Specific Criteria Name Serves Triple Purpose

The Rule: When staff creates chapter-specific criterion (e.g., "Q1 Marketing Forum"), the name serves as: (1) Column header in reports, (2) ME Classification name for assignment to events/committees/groups, (3) Unique identifier in system.

The Gotcha: Editing criterion name changes the ME Classification name. If events already have "Q1 Marketing Forum" ME Classification assigned, and staff renames criterion to "Q2 Marketing Forum", those event assignments may become orphaned or behavior may change.

QA Test: Create criterion "Q1 Marketing Forum". Assign ME Classification to an event. Rename criterion to "Q2 Marketing Forum". Verify event ME Classification also updates, and participants are still scored correctly.

🏁 Rule 6: Min # = Minimum Occurrences Before Points Awarded

The Rule: Min # field specifies minimum occurrences required before points are awarded. If Min # = 3 and member has 2 occurrences, zero points awarded (threshold not met).

The Gotcha: Min # defaults to 1 (typically). If staff changes Min # after participants have already accumulated occurrences, the threshold retroactively applies. Example: criterion starts with Min # = 1 (member earns points after 1 event), staff changes to Min # = 3. Member who attended 1 event now scores zero until they attend 2 more.

QA Test: Create criterion with Min # = 1. Register member for 1 event. Verify 1 point awarded. Change Min # to 3. Regenerate report. Verify member now scores zero until 2 more occurrences.

🔄 Rule 7: Multiple Occurrence Flag Controls Accumulation

The Rule: When Multiple Occurrence is UNCHECKED, points are awarded once regardless of occurrence count. When CHECKED, points accumulate per occurrence.

The Gotcha: If criterion has Point Value = 10 and Multiple Occurrence is UNCHECKED, attending 3 events = 10 points total (not 30). If CHECKED, attending 3 events = 30 points total.

QA Test: Create two identical criteria: one with Multiple Occurrence OFF, one ON. Point Value = 10 for both. Register member for 3 events with both classifications. Verify first criterion shows 10 points, second shows 30 points.

🏷️ Rule 8: ME Classification Is SEPARATE from Standard Classification

The Rule: ME Classification is a dedicated field used only for engagement reporting. Standard Classification (used for event type, registration rules, etc.) is independent.

The Gotcha: Event can have Standard Classification "Annual Conference" (controls event behavior) and ME Classification "Attend Mgmt Ed Events" (controls engagement scoring) simultaneously. If staff confuses these, they may assign wrong ME Classification, breaking engagement logic.

QA Test: Create event with Standard Classification "Annual Conference". Assign ME Classification "Attend Mgmt Ed Events". Register member. Verify member scored for "Attend Mgmt Ed Events" engagement, not "Annual Conference". Verify event type behavior (e.g., registration restrictions) applies normally.

📊 Rule 9: ME Classification Determines Engagement Contribution

The Rule: When an event, committee, or group is assigned an ME Classification, any participant in that entity receives points for that classification's associated criterion (if activated).

The Gotcha: If event has ME Classification "Committee Participation" (wrong assignment), committee members who attend this event would be scored for committee participation, which is incorrect.

QA Test: Create two events: Event A with ME Classification "Attend Membership Events", Event B with ME Classification "Committee Participation". Register same member for both. Verify member scores correctly for each classification in engagement report.

🔐 Rule 10: Standard Criteria Have Read-Only Subsystem Indicators

The Rule: Each standard criterion has checkboxes for Event, Committee, Group subsystems. These are read-only—chapter staff cannot modify. They indicate which subsystems that criterion applies to.

The Gotcha: Staff may expect to customize subsystem indicators but cannot. If a standard criterion applies to Events but staff needs it for Committees only, staff cannot modify the indicator—they must create a custom chapter-specific criterion instead.

QA Test: View standard criteria in configuration. Verify subsystem checkboxes are read-only (no edit option). Verify chapter-specific criteria allow customization of subsystem checkboxes.

Rule 11: Company Total Points = Company Level + Individual Sum

The Rule: In company-level reports, company total points = (company-level points, if applicable) + SUM(all individual member points from that company).

The Gotcha: If company has 5 members with 100, 150, 200, 75, 50 points respectively, company total = 575 points (assuming no company-level points). Staff may expect company to have independent score, not aggregate.

QA Test: Create company with 5 members. Assign each member different engagement scores. Generate company-level report. Verify company total = sum of individual totals. Test with company-level points (if applicable) + individual sum.

👥 Rule 12: Individual Points Grouped by Company in Individual Output

The Rule: Individual-level reports display individual member scores, grouped by company. Example: Company ABC (3 individuals), Company XYZ (2 individuals).

The Gotcha: Grouping must be clear and sortable. Staff may want to export individual-level data and filter by company in Excel. Verify grouping is preserved in export.

QA Test: Generate individual-level report. Verify individuals are grouped under company names. Test sorting (by company name, then individual name). Export to Excel and verify grouping persists.

💾 Rule 13: Configuration Persists Across Sessions; Report Filters Do Not

The Rule: Configuration settings (Standard Criteria activation/deactivation, chapter-specific criteria, ME Classifications) are saved and persist. Report filters (date range, company filters, sorting) are applied per-run and do not automatically persist to next session.

The Gotcha: Staff may create a custom report with specific filters, close browser, then expect filters to reapply. They will not. Staff must re-enter filters each time.

QA Test: Configure custom criterion and assign ME Classification. Close browser. Log back in and verify custom criterion still exists. Generate report with specific filters. Close and reopen. Verify filters are reset to defaults, not saved.

👁️ Rule 14: Inactivating Criteria Retains ME Classification Assignments

The Rule: When chapter-specific criterion is inactivated, events/committees/groups that have that criterion's ME Classification assigned retain the assignment. ME Classification remains on entity but criterion is not scored (hidden).

The Gotcha: If staff inactivates criterion and later reactivates, scoring resumes using same ME Classification assignments. Historical data is preserved.

QA Test: Create criterion "Q1 Marketing Forum", assign ME Classification to event, inactivate criterion. Run report and verify criterion is not scored. Reactivate criterion and regenerate report. Verify scoring resumes for that event.

👁️‍🗨️ Rule 15: Hide Inactive Toggle Controls Visibility of Chapter-Specific Criteria

The Rule: Default ON. Hides inactive chapter-specific criteria from configuration UI. Staff can toggle OFF to view full list including inactive. Standard criteria cannot be hidden.

The Gotcha: Toggle affects UI visibility only, not report calculations. Inactive criteria still calculate if data exists. Staff may get confused if they hide criteria, forget it's hidden, and search for it later.

QA Test: Inactivate custom criterion. Verify it disappears from list (toggle ON). Toggle OFF and verify it reappears. Verify report still calculates for inactivated criterion if data exists.

📈 Rule 16: Report Results Sortable by All Columns; Default Sort by Total Points Descending

The Rule: Report results allow sorting by any column (company name, individual name, total points, specific criteria, paid status). Default sort is by Total Points descending (highest engagement first).

The Gotcha: Sorting must be stable and performant, especially with large datasets (100+ companies). Staff may apply multiple sorts (company, then points) and expect results to maintain logic.

QA Test: Generate company-level report. Verify default sort by Total Points descending. Click column headers and verify sorting applies. Test multi-column sorting if UI supports.

📥 Rule 17: Export to Excel Available for Both Company and Individual Outputs

The Rule: Both company-level and individual-level reports can be exported to Excel format. Export preserves all columns and applied sorting/filtering.

The Gotcha: Export file name should be descriptive (e.g., "ME_Report_2024_CompanyLevel.xlsx") and timestamp may help with version control. File format should be .xlsx (Excel 2007+).

QA Test: Generate report, apply filters/sorting, export to Excel. Verify file downloads with appropriate name, format, and content. Spot-check totals and data match on-screen display.

🌟 Rule 18: Sponsor Criteria Has Dual Path

The Rule: "Sponsor Event" standard criterion can be satisfied in two ways: (1) via event sponsor tab (if event has sponsor field), or (2) via event with "Sponsor" ME Classification assigned.

The Gotcha: Both paths may apply to same event, creating potential for double-counting if not handled correctly. System must prevent double-awarding points for same sponsorship action.

QA Test: Create event with sponsor tab data AND "Sponsor" ME Classification. Register sponsor. Verify points awarded once, not twice. Test with sponsor tab only, ME Classification only, and both.

QA Focus for Business Rules & Gotchas

These 18 rules define the behavior boundary of Member Engagement Reporting. Each rule has a corresponding gotcha—a likely misunderstanding or edge case. Use these gotchas to design test cases that verify the system works correctly even when behavior is non-obvious.

Scenario Thinking

Real-world test scenarios that combine multiple features and workflows. These scenarios drive comprehensive end-to-end testing.
🆕 Scenario 1: New Chapter Setting Up ME Reporting

Setup: Chapter is implementing Member Engagement Reporting for the first time in 2024.

Workflow: Staff navigates to ME Reporting configuration. Reviews 17 standard criteria and decides which to activate (e.g., deactivates "STEP" and "AQC" as not relevant to chapter). Creates 3 custom criteria: "Attended Q1 Forum", "Served on Board", "Donated to Scholarship Fund". Assigns Point Values and Min # thresholds. Assigns ME Classifications to existing events, committees, and groups. Generates first engagement report for Jan-Dec 2024.

QA Validation: Verify all configuration steps complete without error. Verify first report generates with correct criteria included. Verify company and individual point calculations are accurate. Verify no data loss or orphaned references.

🔄 Scenario 2: Chapter Migrating from Manual Tracking

Setup: Chapter has been manually tracking engagement in spreadsheets. Now importing 2 years of historical data into ME Reporting.

Workflow: Staff sets up ME Reporting configuration for 2023 and 2024 years. Imports historical participation data (event attendance, committee membership, donations). Runs engagement reports for both years and compares against historical spreadsheets to validate accuracy.

QA Validation: Verify historical data import completes without loss. Verify reports match historical tracking (within tolerance). Verify configuration supports multi-year reporting.

✏️ Scenario 3: Mid-Year Criteria Changes

Setup: Chapter is mid-year (July) when executive decides to adjust engagement scoring (increase point values for leadership roles, add new "Young Professional" focused criterion).

Workflow: Staff increases "Leadership Committee" point value from 10 to 15 points. Creates new custom criterion "Young Professional Participation" worth 5 points. Assigns ME Classification to existing young professional events. Generates updated engagement reports for Jan-Dec.

QA Validation: Verify point value changes apply retroactively to Jan-Jun data. Verify new criterion only scores from July forward (or all year if staff assigns retro). Verify company totals recalculate correctly.

🏢 Scenario 4: Company with Zero Individual Participation

Setup: Large company account with multiple individual members but zero participation in events/committees/groups during reporting period.

Workflow: Staff generates engagement report. Company appears in report with zero engagement points. Individual member list shows all members but each with zero points.

QA Validation: Verify company appears in report (not filtered out). Verify zero points display correctly. Verify staff can still target this company for outreach using report.

🌟 Scenario 5: Individual Member with Activity Across Multiple Criteria

Setup: Highly engaged individual who attends events, serves on committee, donated, and participated in leadership program.

Workflow: Individual's participation is scattered across events (Attend Membership Events: 3 points × 5 events = 15 points), committee (Committee Participation: 10 points × 2 meetings = 20 points), donation (Voluntary Contributions: 25 points), and leadership (Leadership Committee: 15 points). Total expected: 75 points. Staff generates report to identify high performers for recognition.

QA Validation: Verify all criteria contributions are summed correctly (15 + 20 + 25 + 15 = 75). Verify individual appears at top of report (sorted by total points descending). Verify company total includes this individual's 75 points.

📊 Scenario 6: Large Report Generation (100+ Companies)

Setup: Chapter with 150 member companies and 2,000+ individual members. Staff generates full engagement report for all.

Workflow: Staff navigates to Member Engagement Reporting, sets year to 2024, does not apply company filters, and clicks "Generate Report". System processes data and displays 150 company rows with individual breakdowns. Staff applies sorting (by total points descending) and exports to Excel.

QA Validation: Verify report generates in reasonable time (under 30 seconds for typical infrastructure). Verify all 150 companies display. Verify sorting applies without errors. Verify Excel export includes all data and is usable in Excel (no corruption). Verify no data loss or missing rows.

🤔 Scenario 7: ME Classification vs Standard Classification Confusion

Setup: Staff creates new event "Annual Leadership Summit" and must assign both Standard and ME Classifications.

Workflow: Staff assigns Standard Classification "Conference" (controls event type, registration rules, pricing). Then assigns ME Classification "Leadership Committee" (controls engagement scoring). Staff may confuse the two and assign wrong ME Classification (e.g., "Annual Conference" instead of "Leadership Committee").

QA Validation: Verify UI clearly distinguishes Standard vs ME Classification fields. Verify help text explains difference. Test that registrants are scored for correct ME Classification regardless of Standard Classification. Test with wrong ME Classification assignment to ensure it doesn't break engagement logic (should still score, just under wrong criterion).

♻️ Scenario 8: Inactive Criteria Reactivation with Historical Data

Setup: Chapter inactivates custom criterion "Q1 Marketing Forum" in March. Later in year, decides to reactivate it to include in annual report.

Workflow: Staff inactivates "Q1 Marketing Forum" (criterion hidden, no scoring). Reactivates it in November. Generates year-end engagement report with "Include Annual Criteria" ON.

QA Validation: Verify inactivated period excluded from scoring (Jan-March: no points). Verify reactivation period included (Nov-Dec: points awarded). Verify historical data from Jan-March is preserved but not scored during inactive period. Verify year-end report correctly reflects only Nov-Dec scoring for this criterion.

QA Focus for Scenario Thinking

These scenarios combine multiple features and real-world workflows. Use them to design end-to-end test cases that validate Member Engagement Reporting in realistic contexts. Each scenario reveals potential failure points and integration issues.

Regression Checklists

Regression testing checklists organized by functional area. Use these to ensure changes to one area don't break existing functionality.
Configuration Regression Checklist
  • Year selection auto-populates correct date range (1/1 to 12/31)
  • Date range can be manually edited without validation errors
  • Company filters (paid status, size, etc.) apply correctly
  • Standard criteria activation/deactivation toggles and persists
  • Chapter-specific criteria can be created with all fields
  • Chapter-specific criteria can be edited without losing data
  • Chapter-specific criteria can be inactivated and reactivated
  • Hide Inactive toggle shows/hides inactivated criteria correctly
  • ME Classifications can be assigned to events/committees/groups
  • ME Classification assignment persists across sessions
  • Configuration changes do not clear previous settings unless explicitly intended
  • Point values, Min #, and flags save correctly for all criteria
📈 Report Calculation Regression Checklist
  • Company total = company-level points + sum of individual member points
  • Individual points calculated correctly based on ME Classifications
  • Multiple Occurrence flag controls accumulation (single vs. per-occurrence)
  • Min # threshold enforced (points not awarded if occurrences < Min #)
  • Annual Designation criteria included when "Include Annual Criteria" checkbox ON
  • Annual Designation criteria included regardless of date range when annual flag set
  • Non-annual criteria filtered by date range (outside range = no points)
  • Zero participation results in zero points (not error)
  • Point values correctly multiplied by occurrence count when Multiple Occurrence ON
  • Mid-year criteria changes retroactively apply to historical data
  • Inactive criteria excluded from scoring (no contributions to totals)
  • Company and individual level totals match between report views
🏷️ Classification Regression Checklist
  • ME Classification is independent of Standard Classification
  • Event can have both Standard and ME Classification without conflict
  • Committee can have both Standard and ME Classification without conflict
  • Group can have both Standard and ME Classification without conflict
  • ME Classification assignment to entity doesn't affect standard classification behavior
  • ME Classification on events drives correct scoring in engagement reports
  • ME Classification on committees drives correct scoring
  • ME Classification on groups drives correct scoring
  • Participants in entity with ME Classification receive correct points
  • Changing ME Classification on entity updates scoring for future reports
  • ME Classification names match Standard or Chapter-Specific Criteria names
  • No double-scoring or missed scoring due to classification confusion
📤 Export Regression Checklist
  • Excel export includes all columns from report view
  • Excel export includes all rows (no truncation)
  • Excel export respects applied sorting (matches on-screen order)
  • Excel export respects applied filters (exported data matches filtered view)
  • Excel file is valid and opens without corruption in Excel/Sheets
  • Number formatting is correct (currency, decimals, thousands separators)
  • Column headers are clear and match on-screen labels
  • Company-level export includes all companies and correct totals
  • Individual-level export includes all individuals and grouping by company
  • Exported file has descriptive name with date/timestamp
  • Export completes without timeout for large datasets
  • Spot-check exported data matches on-screen displayed values
🎨 UI Regression Checklist
  • ME Reporting module accessible from main navigation
  • Report Settings section displays all fields and labels
  • Standard Criteria section lists all 17 pre-defined criteria
  • Chapter-Specific Criteria section allows create/edit/inactivate actions
  • ME Classifications can be assigned to Events, Committees, Groups
  • Report generation button triggers report display
  • Company-level and individual-level report options both available
  • Sort controls visible and functional on all columns
  • Filter controls available for date range, company status, and criteria
  • Export button present and functional
  • Help text and tooltips explain ME Classification vs Standard Classification
  • Standard criteria marked as "System-Defined" or similar to prevent user deletion
  • Inactive criteria visually distinguished (grayed out, label, etc.)
  • Layout responsive on desktop and mobile (if applicable)
QA Focus for Regression Checklists

Use these checklists as regression test suites for each code deployment. Run these checklists after any changes to configuration UI, report calculation logic, classification behavior, or export functionality to ensure no regressions.

Environment & Data Setup

Test environment configuration, required data, and setup instructions for comprehensive testing.
🔗 Environment Details

URL: https://chad2qahub.ba.orases.dev

Environment: QA/Test environment for CHAD2 system

Access: Requires QA user account with appropriate permissions (Chapter L2/L3 access to ME Reporting module)

Data Reset: QA environment may be reset periodically. Coordinate with QA team for data continuity needs.

📋 Test Data Requirements

Companies: Setup test data with variety of membership statuses:

  • Active, paid company (5 members)
  • Active, unpaid company (3 members)
  • Inactive company (2 members)
  • Large company (20+ members)
  • Single-member company

Individuals: Create individuals with varying participation levels:

  • Highly engaged (participates in 5+ events, 2+ committees)
  • Moderately engaged (3-4 events, 1 committee)
  • Minimally engaged (1-2 events)
  • Not engaged (no participation)

Events: Create events with ME Classifications assigned:

  • 5+ membership events with "Attend Membership Events" ME Classification
  • 3+ educational events with "Attend Mgmt Ed Events" ME Classification
  • 2+ leadership events with "Leadership Committee" ME Classification
  • 2+ events with "Sponsor Event" ME Classification
  • Events spanning Jan-Dec for full-year testing

Committees: Create committees with ME Classifications:

  • 3+ committees with "Committee Participation" ME Classification
  • 2+ committees with "Leadership Committee" ME Classification
  • Committee membership that includes test individuals

Groups: Create groups with ME Classifications (if applicable)

🛠️ Setup Instructions

Step 1: Environment Access

  1. Navigate to https://chad2qahub.ba.orases.dev
  2. Log in with QA user credentials
  3. Verify ME Reporting module is accessible from navigation

Step 2: Configuration Setup

  1. Navigate to ME Reporting configuration
  2. Review and activate/deactivate standard criteria as needed for testing
  3. Create 3-5 custom chapter-specific criteria for testing
  4. Assign ME Classifications to existing events, committees, and groups

Step 3: Test Data Population

  1. Use existing test companies or create new ones as needed
  2. Ensure individuals are assigned to companies with varying participation
  3. Populate participation data: individuals registered for events, committee membership, donations, etc.
  4. Verify participation data flows correctly to ME Reporting module

Step 4: Report Baseline

  1. Generate first engagement report with baseline configuration
  2. Document baseline report values for regression comparison
  3. Export to Excel for offline verification
Pre-Test Verification Checklist
  • QA environment accessible and responsive
  • ME Reporting module visible in navigation
  • Test user has L2/L3 access (can configure criteria)
  • Test companies created with correct membership status
  • Test individuals created and assigned to companies
  • Test events created and ME Classifications assigned
  • Test committees created and ME Classifications assigned
  • Participation data (event registrations, committee membership) populated
  • Standard criteria list displays all 17 criteria
  • Chapter-specific criteria creation functional
  • Configuration settings save and persist
  • Initial report generation completes without error
  • Report displays expected companies, individuals, and point calculations
  • Export functionality works (test with small report)
QA Focus for Environment & Data Setup

Proper environment and test data setup is critical for comprehensive testing. Invest time in creating diverse, realistic test data that exercises all code paths. Ensure baseline reports are documented for regression comparison.